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Maturation of intelligent systems technologies and their incorporation into aerial 
platforms are dictating the development of new analysis tools and incorporation of such tools 
into existing system analysis methodologies in order to fully capture the trade-offs of 
autonomy on vehicle and mission success. A first-order “system analysis of autonomy” 
methodology is outlined in this paper. Further, this analysis methodology is subsequently 
applied to notional high-altitude long-endurance (HALE) aerial vehicle missions. 


Nomenclature 


a x Coefficients, defined in text 
Alt Altitude 

c t Percent coverage of target in final data product 
(“image”) 

c x Coefficients, defined in text 
C A s Autonomous System Complexity 
C E o Environmental and Operational Complexity 
C G cs Cost of ground control station 
C m Cost of mission (if mission prematurely 
terminated) 

C v Cost of vehicle (if unexpectedly lost) 

D c “Discovery confidence” metric for aerial 
explorers 

Dj “Degree of inaccessibility” 

D in “Degree of interaction” among UAVs and other 
intelligent systems to complete a given mission 
D r “Degree of resistance” 

F P Processor speed (instruction or operation) 

F t Target frequency 
GW Gross Weight 

hi Time by i th person on considered tasks, in hours 


LO A Level of Autonomy 

m Fraction of mission completed prior to premature 
termination 

m x Coefficients, defined in text 

M Baseline “staffing” matrix 

n Number of operations personnel 

n x Coefficients, defined in text 

N b Number of robotic behaviors 

N c Number of control actuators 

N DO f Number of degrees of freedom, robot mobility 

N l Lines of software code 

N m Size of system dynamic memory 

N P Number of processors 

N r Number of robots (and/or intelligent systems, 
including autonomous aerial vehicles) 

NR U ie Number of conditional (heuristic) rules 
N s Number of sensors 
N v Number of state space variables 
o(t) Obsolescence of data product upon delivery 
(hours) 

P MTBF (mean time between failure) x mission 
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duration 

q Production quantity of HALE aircraft of a 
particular type/mission 

Q QFD-inspired technology-to-goals matrix 
r t Resolution adequacy of target image (0 to 1) 

R Factor for consequence of failure; values for types 
of events: return to base = 0; damage to aircraft = 
(cost to repair)/(vehicle cost); crash in 
unpopulated area = 2 (loss of vehicle plus 
retrieval costs); crash into urban area =100 
R c Control input rate 
MROI Mission return on investment 
s t Clarity adequacy of target image (0-1) 

S Mission success 

t Specific target 

T Mission duration 

T 0 Set of observation targets defined by users (as 
applied to wildfire surveillance missions) 

T M i i'th case of M mission tasks (for multiple 
coordinated flyers) 

v t Value from perfect observation of target t 
Wi Wage of i th person, $/hr 


oti Weight factor reflecting importance of overall 

mission task (as executed by multiple, coordinated 
flyers) 

Pi success of individual task, (as executed by 
multiple, coordinated flyers) 
y Discount factor for data obsolescence (percent 
loss value per hour) 
i Aerial vehicle intelligence metric 

s Aerial vehicle autonomous system 

implementation elegance metric 
Telecom relay messages per aircraft per day 
Station-keeping science observations per day 
K Level of Autonomy (LOA), 0<K<5, “aleph” 

Superscript: 

* “Normalized” metrics of vehicle intelligence, 
elegance, and degrees of inaccessibility, resistance, 
and interaction, such that range of value of 
parameters fall within 0 and 10. 


Introduction 

SYSTEM analysis is an essential technical discipline for the modem design of aerial vehicles and their 
associated missions. Specifically, system analysis is a powerful aid in identifying and prioritizing the required 
technologies needed for mission and/or vehicle development efforts. Maturation of intelligent systems technologies, 
and their incorporation into aerial platforms, is dictating the development of new analysis tools, and incorporation of 
such tools into existing system analysis methodologies, in order to fully capture the tradeoffs of autonomy on 
vehicle and mission success. A first-order “system analysis of autonomy” methodology will be outlined in this 
paper. Further, this analysis methodology will be applied to a set of notional high-altitude long -endurance (HALE) 
uninhabited aerial vehicle (UAV) missions. 

For planetary flight vehicles (PFV), a high level of autonomy is essential to accomplish their missions. The 
substantial time delay in interplanetary communications makes aerial vehicle autonomy a necessity. For terrestrial 
uninhabited aerial vehicles (UAV), though, autonomy (above that of teleoperation, or rather simple remotely piloted 
vehicle capability) is not essential but, nonetheless, could be a powerful enabling technology. Other than the most 
qualitative of assessments, how can cost/benefit analysis for incorporation of autonomous system technologies into 
aerial vehicles be performed? The first step needs to be the definition of metrics for autonomy that are indeed 
unambiguous and quantifiable. 

Fundamental concepts such as autonomy , intelligence , and elegance for planetary flight vehicles (a.k.a. aerial 
explorers) have been earlier presented 1 . In many respects, these autonomy metrics needed to be uniquely tailored 
for aerial explorers. In the case of terrestrial UAVs, previously proposed 2 autonomous control levels (ACL) have 
been more generally accepted. The NASA Vehicle Systems Program (VSP) HALE Sector concluded that the 
DOD/AFRL ACL were too extensive (too many and too nuanced) for the envisioned HALE science missions. A 
simplified set of level of autonomy (LOA) metrics have therefore been adopted by the VSP HALE Sector. 

Once having defined quantifiable autonomy metrics for UAVs, reasonable first-order functional relationships are 
defined relating vehicle and mission characteristics to aforesaid autonomy metrics, and ultimately related to 
individual autonomous system technologies. Hopefully, this preliminary “system analysis for autonomy” 
methodology, though still somewhat conjectural, will serve as a starting point for a robust future discussion of an 
important nascent area of engineering investigation. 
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A. Autonomy - keep it simple 


What is this “autonomy” all about? It is about taking the human out of the airplane. Not completely out of the 
loop, but minimizing human involvement in the operation of the vehicle while providing enhanced mission 
flexibility and vehicle robustness. UAVs will be operated for durations exceeding the physical capabilities of a 
human operator and the cost of operating the vehicle can be minimized by reducing the work load required to a level 
sufficient that numerous aircraft may be operated by a single operator. The AIAA defines autonomy as “A systems 
own ability of sensing, perceiving, analyzing, communicating, planning, decision-making, and acting, to achieve its 
goal as assigned by human operators”. 23 However this does not provide the perspective for collaborative vehicle 
operations where UAVs will eventually excel. To address this issue numerous organizations have attempted to 
define autonomy in terms of a Level of Autonomy (LOA). In most cases the proposed LOA become complicated 
and address vehicle capabilities beyond those needed for typical science and civil applications. Since no readily 
applicable LOA definition existed that exactly matched the VSP HALE Sector envisioned operational aspects of 
HALE UAVs, the Sector established its own definition. 

Table 1 defines the LOA used by the HALE Sector to articulate its autonomy goals and objectives. Important in 
the definition of the LOA is the concept of “hands-on-time”. Hands-on-time is defined as the percentage of a pilot’s 
time (direct attention paid to the UAV) that would be required to safely operate the UAV during a given mission. 
Hands-on-time is limited to the pilot/operator and does not include other mission personnel (e.g., sensor/payload 
operator). Nominal values are presented in the table, actual hands-on-time will vary with the mission. 

What are the primary autonomy enablers? To be able to fly in civil airspace you need an UAV system with a 
demonstrated Equivalent Level of Safety (ELOS) as a manned system. It cannot cost any more to operate than a 
manned system. In fact, one should be able to realize a cost reduction through automation. The primary challenges 
(hurdles) to achieving these fundamental desired features are: system safety and reliability (addressing 
airworthiness, sense & avoid, and validation & verification); fault tolerant system architectures (providing 
robustness, situational awareness, automated operations); contingency management (fault detection, prognostics, 
emergency procedures); and other miscellaneous factors such as software certification, ELOS certification, and Air 
Traffic Management (ATM) interfaces with manned and other unmanned assets. 

B. Various Notional HALE Missions and Overall Characteristics 

The NASA VSP is in the process of completing a survey of potential civilian UAV missions - including High 
Altitude Long Endurance (HALE) science platforms. 16 Some of the notional missions are summarized in Table 2. 
These representative missions will be referred to in later sections of the paper - particularly in the context of 
benchmarking UAV autonomy metrics and establishing functional relationships for the proposed “system analysis 
for autonomy” methodology. 
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Table 1. Levels of Autonomy adopted by the VSP HALE Sector 


LOA 

Level 

Description (Features) 

Sample Characteristics 

0 

Remote 

Controlled 

Remotely piloted aircraft with a human in the loop, making all 
the decisions. Operator is in constant control. (100% hands- 
on-time.) 

> R/C airplane 

1 

Simple 

Automation 

Remotely piloted with some automation techniques to reduce 
pilot workload. Human monitoring to start/stop tasks. (80% 
hands-on-time.) 

> Basic autopilot 

2 

Remotely 

Operated 

Human operator allows UAV on-board systems to do the 
piloting. As part of the outer control loop, the human makes 
decisions as to where to go, when, what to do once there. 
Remotely supervised, with health monitoring and limited 
diagnostics. Operator allows UAV to execute preprogrammed 
tasks, only taking over if the UAV is unable or fails to 
properly execute them. (50% hands -on-time.) 

integrated Vehicle 
Health Management 
(IVHM) 

> Onboard Contingency 
Management 
capabilities 

> Waypoint navigation 

3 

Highly- 

Automated 

or 

Semi- 

Autonomous 

UAV automatically performs complex tasks. System 
understands its environment (situational awareness) and 
makes routine decisions and mission refinements to 
dynamically adjust to flight and mission variables. Limited 
human supervision, managed by exception. Adaptive to 
failures and evolving flight conditions. (20% hands-on-time.) 

> Loss-link mission 
continuation 

> Automatic takeoff/land 

> Adaptive control 
techniques 

> Reactive “search and 
find” terrain recognition 

4 

Fully 

Autonomous 

UAV receives high-level mission objective (e.g., location, 
time), translates them into tasks that are executed without 
further human intervention. UAV has the ability and 
authority to make all decisions. Extensive situational 
awareness (internal and external), prognostics, and on-board 
flight re-planning capability. Single vehicle operations. (Less 
than 5% hands -on-time.) 

> Automated in-flight re- 
planning 

> Mission sensor-directed 
operations 

5 

Collaborative 

Operations 

Brings in aspects of multiple UAVs working autonomously 
together as a collective intelligent system. Group 
coordination. Individual vehicles/sy stems in a collaborative 
group will have a least semi-autonomous LOA (3) to keep the 
operator workload of the collaborative operation at a 
manageable level. (Total hands-on-time for sum of all air 
vehicles would not exceed a single operator hands-on-time of 
100%.) 

> Cooperative and 
collaborative flight 

> Mother- and daughter- 
ship collaborative 
operations 

> Team leader concept for 
cooperative systems 

> Robotic swarms 
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Table 2. Representative (Notional) HALE Missions 

Telecom Relay Platform Mission/fight profile : transit from local commercial airport to station-keeping altitude 
(>60K ft), long duration loiter (>30 days). Critical capability : flight endurance and payload capacity. Technical 
challenges : power and propulsion; reliability. Autonomous system technology challenges : highly autonomous 
operations during station-keeping, autonomous payload management, and health monitoring and prognostics of 
flight systems. Automated piloting to and from station-keeping altitude. Risk/hazard : disruption of services. 
Severe Storm-Tracker Mission/fight profile : launch/recovery from regional centers; mid-altitude 
(40K<Alt<60K ft) long-range (>1000 km) cruise to operational area; increase altitude (>60kft) and engage/track 
storm for duration (4 to 10 days). Critical capability : robust structural and aero-handling characteristics to fly in 
moderate turbulence. Autonomous system technology challenges : adaptive flight controls (e.g., aero-handling, 
potential control surface failure); science payload management; sensor/payload-driven in-flight re-planning (e.g., 
onboard lidar/radar tracking of immediate-vicinity storm patterns and velocity flow-fields) for flight path 
management and goal-based decision-making. Risk/hazard: loss of aircraft due to severe weather conditions. 
Border & Coastal Patrol Mission/fight profile : launch/recovery from a regional center, cruise at mid-altitude 
(40K<Alt<60K ft), long-range (>1000 km), and moderate duration (4 to 10 days). Critical capability : stable flight 
platform with high-resolution imaging/sensing capability and ability to traverse large survey areas. Autonomous 
system technology challenge : science payload management; real-time re-planning; secure, high-bandwidth data 
relay/networking. Risk/hazard : inability to acquire and transfer critical intelligence to appropriate decision- 
makers in a timely manner. See Ref. 22. 

Station-Keeping Science Platform - observation of persistent science phenomenon; e.g., monitoring volcanoes, 
polar “ozone holes,” environmental effects of chemical spills and toxic ocean “blooms,” etc. Mission/flight 
profile : deploy from national science centers, transit to station-keeping altitude (>60K ft), long duration loiter 
(>30 days). Critical capabilities : long endurance, low-speed loiter capability coupled with large payload capacity. 
Autonomous system technology challenge : automated mission operations during station-keeping including 

science payload management. Risk/hazard: disruption of services. 

Long-Range Science & Surveillance Platform - observation of transient and/or widely-distributed 
phenomenon; e.g., damage surveys and coordination for large-scale disasters such as earthquakes and hurricanes; 
tracking migration patterns of animals/insects; mapping/measuring the spatiotemporal variation of air and ocean 
currents; assessing compliance of international laws/regulations over international waters, e.g. prohibition on 
whale harvesting. Mission/flight profile : long-range (>1000 km), moderate duration (4 to 10 days), moderate 
altitude (40K<Alt<60K ft) cruise capability; navigation by pre-planned GPS waypoints and sensor/science-driven 
decision-making. Autonomous system technology challenges : sensor/science-driven navigation and goal -based 
decision-making. Risk/hazard : inability to acquire and transfer critical data/information to appropriate 
users/decision-makers in a timely manner. 

Sensor-Network Aerial Constellation - repeat pass interferometry (RPI), military and civilian search and find 
(complementing manned rescue aircraft), dropsonde deployment. Mission/fight profile : multiple UAVs 
conducting coordinated (simultaneous) flights to achieve single mission goal. Critical capability : near-real-time 
or better fusion of data/information from multiple aircraft (and other assets). Technical challenge : repeated 
precision over-flight of critical waypoints/trajectories. Autonomous system technology challenge : vehicle-to- 
vehicle, vehicle-to-ground-control, and vehicle-to-other-intelligent-systems coordination (and in some cases 
“negotiation”). Risk/hazard: operational difficulties in managing multiple assets; hazard/collision avoidance. 
Wildfire Surveyor - satellite monitoring of critical areas is used to pre-plan and launch UAV assets from a few 
national/regional centers to perform real-time monitoring of fire propagation as well as provide coordination 
information for firefighting ground crews and manned aircraft; Mission/fight profile : high-speed (>100 knots) 
mid- to high-altitude flight (40K<Alt<60K ft), long-range capability (>1000 km); short duration (1 day). Critical 
capability : real-time re-tasking/re-planning. Technical challenge : cost-effective on-demand asset 
reliability/availability. Autonomous system technology challenges : intelligent mission management; data & 
information networking. Risk/hazard : inability to acquire and transfer critical intelligence to appropriate 
decision-makers in a timely manner. 

Aerial Explorers - planetary aerial surveys, surface interactive through drop probes and small robotic devices, 
and possibly vertical take-off and landing (VTOL) capability. Non-GPS navigation required. See reference 1. 


5 

American Institute of Aeronautics and Astronautics 




By way of illustration, Fig. 1 is a notional concept of a 14-day endurance HALE platform being studied by the 
NASA HALE Sector. Figure 2 is a notional concept of a 100-day endurance vehicle. The two notional HALE 
flyers also represent a spectrum of mission capabilities, vehicle design, and technology challenges. Both of these 
HALE platforms could satisfy a number of the Table 2 missions. The autonomous system technology challenges 
will be the primary focus of discussion as related to these vehicles and their range of mission applications. 



Figure 1. 14-day HALE Concept (Depicting over-flight of polar region) 



Figure 2. 100-day HALE Concept (Depicting over-flight of hurricane) 


6 

American Institute of Aeronautics and Astronautics 



C. Fundamental Questions about UAV Autonomy 


In respect to HALE UAV autonomy, there are several fundamental vehicle autonomy questions that need to be 
addressed by system-analysis-type trade studies. Among these questions are: 

• Is there a way to identify “breakthrough” UAV missions/applications where autonomy is a key 
technology enabler? 

• How much, and what type/level of, autonomy is appropriate for certain missions and applications? 

E.g., is there a need for autonomous capability greater than teleoperation (i.e. remote-piloting) for 
HALE UAV? 

• For a given mission/application and appropriate associated autonomy level, what is the minimum 
required level of intelligence and elegance (efficiency of the intelligence implementation/realization) 
to yield acceptable mission success, cost, and reliability thresholds? What, if any, are the negative 
consequences of building in too much autonomy and intelligence into an aerial vehicle? 

Ancillary questions to the above cost/benefit question can be posed for certain intelligent sub-systems 
above and beyond the vehicle autonomy implementation itself. 

• How much cumulative “autonomous system technology” development cost is acceptable before 
human operation is more cost effective? 

• What is the breakeven cost for development implementation of intelligent onboard 
prognostics/diagnostic systems, given certain assumed missions? And, further, what is the 
breakeven cost of development and implementation of intelligent/adaptive flight control systems? 

• What is the cost/benefit trade between autonomous system elegance (computational efficiency and 
avionic system complexity) and vehicle reliability and safety? 


D. HALE UAVs and the Impact of Autonomy 

Autonomous systems technologies find their greatest utility not in just replacing onboard pilots, engineers, and 
scientists with ground-based operators/monitors, but with expanding into new application domains and enabling 
wholly new mission applications. A number of these new mission applications, both terrestrial and planetary, were 
presented in Table 2. However, it is also important to note that the greatest impact of autonomous system 
technology is not limited to its implementation on a single aerial platform, or even multiple versions of a single 
vehicle type, but rather the greatest impact will result from the application of autonomy to heterogeneous robotic 
systems working in cooperation/collaboration with each other to achieve mission success. Figures 3a and 3b show 
two examples of the simplest form of cooperating system of robotic assets: the deployment of “intelligent” 
dropsondes from a terrestrial UAV and mid-air-deployment of “robotic symbiotes” from a planetary aerial explorer. 
Many such coordinated and cooperating systems of robots, including/incorporating UAV aircraft, can and have been 
conceived in principle. In an ideal sense unmanned aerial vehicles do not merely replace manned aircraft, but are in 
fact true aerial robots. 



(a) 


(b) 


Figure 3. Examples of Collaborative Operations: (a) dropsonde from HALE 
planetary aerial explorer and air-deployed robotic symbiote 


‘storm tracker” and (b) 
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I. The Proposed Basic Methodology 


The basic “system analysis of autonomy” methodology is outlined in this section. Figure 4 presents a high-level 
flow chart of the proposed methodology. Each element of the method will be discussed in detail. 



Figure 4. System Analysis Flow Chart 
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A. Statement of Problem 


The problem to be studied is easy to describe but difficult to solve: 

• First, what is the optimum level of vehicle autonomy and intelligence required for a particular HALE UAV 
mission/ application, so as to assure acceptable levels of success and safety while at the same time keeping 
development and implementation costs to a minimum? 

• Second, what are the specific attributes of an autonomous system implementation essential for a given 
mission/ application and aerial vehicle in order to maximize mission success? 

The engineering community must be careful not to imbue systems with unnecessary, or otherwise inappropriate, 
levels of autonomy and intelligence for the particular purposes to which they are applied. To draw on examples from 
science fiction, there is no need for talking toasters a or smart bombs that pose existential questions 13 . Similarly, 
aircraft should not be able to decide that they would rather fly to Palm Springs than Seattle — for the better weather - 
- on any given day or flight. (Such decision-making freedom, though, may well be appropriate for a planetary aerial 
vehicle, so as to enable “serendipitous science” and missions of opportunity.) 


B. Concepts and Initial Metrics 

Considerable work has been documented in the literature as to machine intelligence metrics; however, only 
modest work has been dedicated to UAVs . 20-21 Delving briefly into the autonomy versus intelligence debate, 
autonomy is defined for the purposes of this paper as the ability to independently perform without human 
intervention actions, tasks, or roles. Intelligence measures how well these actions, tasks or roles are performed 
under varying degrees of task and environmental complexity and other associated constraints and conditions. And, 
elegance is the computational efficiency by which the autonomous vehicle intelligence is implemented. Therefore, 
it is wholly possible that two robotic systems can be at nominally equivalent autonomy levels but exhibit radically 
different levels of intelligence. For example, one robot (aerial or otherwise) could perhaps only perform its tasks in 
a simple invariant environment, whereas the other robotic system could perform those nominally same tasks in a 
highly uncertain, unknown, or changing environment. The latter robotic system is clearly more intelligent than the 
robot that can only successfully operate in the simpler environment, though their autonomy levels may be 
equivalent. The following concepts and analysis builds upon earlier autonomous aerial vehicle work . 1,11 " 14,18 

In a simplistic sense, the definitions of intelligence and elegance used in this paper can be expounded upon 
by examining the following relationships, Eqs. (la-d), for a given set autonomy level: 


T ... . _ Environmental & Operational Complexity 

Intelligence = Mission Success x — 

Number of Robots 


Or, alternatively, 


i = 


SCeo 

N r 


Correspondingly, 


Elegance = 


Intelligence 

Autonomous System Complexity x Number of Robots 


a “Red Dwarf’ BBC television show. 
b “Darkstar” cinematic film. 
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Or 


i 

£ = 

r N 

AS iy R 


(la-d) 


Where for autonomy levels 3-5, at least, which is the primary focus of the current work, the following 
holds true: 


Environmental & Operational Complexity 
Robot Mobility Degrees of Freedom 
x Number of Control Actuators 
x Required Control Input Rate 
x Number of Sensors 
x Degree of Inaccessibility 
x Degree of Resistance 
x Degree of Interaction 
x Inverse of Target Frequency 


Or, alternatively, 

C eo =N dof N c R c N s D,D r ^ 

F t 

And 

Autonomous System Complexity 
Lines of Software Code 
x Number of State Space Variables 
x Number of Robotic Behaviors 
x Number of Conditio nal (Heuristic) Rules 
x Number of Processors 
x Size of System Dynamic Memory 
x Mean Processors ) Instruction or Operation Speed 

Or, rather, 


C AS =N L N v N B N Rule N P N M F P 


(2a-d) 

Note that in the expression for “environmental and operational complexity” in Eq. (2a-d) the “degree of 
inaccessibility” term can encompass many things. In its simplest form, it is equivalent to terrain ruggedness, i.e. 
unity plus the variance of terrain elevation obscuring features of interest or impeding mobility (as in nap of the earth 
flight). Also, the “degree of resistance” term can encompass many things, but, again in its simplest form, it is 
equivalent to atmospheric turbulence, i.e., unity plus the variance in wind magnitude and direction as affecting the 
ease of flight control. The “degree of interaction” term encompasses the physical (instruments, dropsondes, robotic 
symbiotes, and other intelligent systems) and informational interactions of the aerial vehicle with objects/features of 
interest on the ground or interactions with external intelligent systems and/or other members of a robotic ecosystem. 
In its simplest form for aerial vehicles it is unity plus the resultant of the number of probes and symbiotes released 
by the aerial vehicle times the post-flight “persistence” of those probes and symbiotes. Finally, the “inverse of target 
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frequency” term, for discrete targets, is given by 7iR^/N Actual (i.e., the inverse of the number of targets within, and 
divided by, a maximum potential survey area described by a circle having a radius equal to the maximum range of 
the aerial vehicle). Note, however, that alternate expressions are required for distributed and scarce targets. 

“Mission return on investment” (MROI) can be expressed as 


, _ T Mission Success 

MROI = 

Risk • Cost 

(3) 

Note that for the special case of planetary science missions 1 - where mission cost is effectively capped, the cost can 
be treated as fixed. 

An earlier discussion of risk and cost has been presented in the context of aerial explorers 18 , particularly as 
related to the question of MSS versus FCL (Many, Simple, Small versus Few, Complex, Large) robotic exploration 
systems. Based on this early work, a measure of mission “return” for a purely scientific endeavor can be defined 
by the relationship (i.e. where mission success is proportional to the amount of information gathered): 


MROI — » Mission Return = 


Information Gathered 
Risk 


(4a) 


Where 


Information Gathered = 

Number of sensors 

X Measure of spatio - temporal dispersal of sensors 
X Mean sophistication of sensors 
X Number of different types of sensors 
X Ability to interprete/adapt given past results 
X Ability to verify results 

X Ability to cross - correlate independent measurements 


And the inverse of risk is 


(4b) 


1/Risk = 

(l - Mean probability of sensor failure ) 

X Mean probability of sensor successful dispersal 
X Mean probability of communicating data back to Earth 


(4c) 

Table 3 notes a number of different measures/metrics for estimating/tracking mission success that can be 
defined. Each mission application domain (niche) has its own unique mission success metric (though there are 
certain common constituent elements). Assessing overall mission success needs to be established through a 
combination of flight test and mission simulation. This will be discussed subsequently in the context of 
autonomous system technology validation. 
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Table 3. Mission Success Metrics for Various Notional Missions 


Mission 

Mission Success Metric 



Telecom Relay Platform 

Mission Success oc ^qT 



Storm-Tracker 

Mission Success oc £c t v t r t s t DjD R D IN T 



Border & Littoral Water Patrol 

Mission Success oc £c t v t r t s t DjT + D c 



Station-Keeping Science Platform 

Mission Success oc £v t r t s t T 



Long-Range Science Platform 

Mission Success cc £c t v t r t s t D IN T + D C 



Sensor-Network Aerial 
Constellation 21 

Nr 

Mission Succes sec Collaborative Gain= ^^cc i P i T M 

/= i 



Wildfire Surveyor 17 

% T~n ^ 

Mission Success = ^,v t c t r t s t ( 1 - y) o{t) - * w ,•) + P • (R • C v + m • C m ) 

t= 1 V i= 1 J 



Aerial Explorers 1 

Intelligence oc Mission Success oc D c oc Targets Found Per Hour 


For PFVs, “mission success” as related to the intelligence metric is directly proportional to the 
“discovery confidence” metric, D c , for aerial explorers working in (robotic) isolation 1 . 

Note that there are no upper bounds on the above definitions of the mission success, intelligence and elegance 
metrics. Their maximum values are dependent upon a given set of vehicle, environmental, and mission 
characteristics — which, in turn, are subject to refinement as vehicle designs are matured, mission concept of 
operations (CONOPS) are detailed, and system characteristics are predicted with improved fidelity 
analysis/simulation tools and/or measured in flight. Though these autonomy metrics can and should be defined by 

using the above detailed formulae, a simpler approach should be used when system analysis of vehicle/mission 

* * 

concepts is to be performed. Therefore “normalized versions of system intelligence and elegance, l and £ , must 
be defined where the normalized values fall within the ranges 0 < i < 10 and 0 < s < 10 . That is, it is posited that 

there are some notional normalization factors such that l = l/l ComtantFactor and £ = £ S ComtantFactor wherein the 
above range of values holds true for all realizable HALE UAV missions. In practice, neither quantitative estimates 
of l and 8, or definition of actual values of l Constant Factor and Constant Factor need be made in 0rder t0 S ain SOme 
advantage from the concept of using normalized measures for these key parameters. The proposed “normalization” 
is a pragmatic technique that allows engineering judgment to be used in the early stages of the system analysis 
process, in place of making detailed but difficult estimates of vehicle intelligence and elegance using Eqs. 1-4. 
Applying representative values for HALE UAVs, the following order of magnitude estimates for the environmental 
and operational and autonomous system complexity can be made: C E0 ~O(10 4 ) and C A s ~O(10 12 ) - assuming that 
the dynamic memory is given in terms of MBytes and the processor speeds in GHz. Correspondingly, the 
normalizing factors are approximately i ConstanlFactor ~ O(l0 3 ) to O(l0 4 ) and e Conslant Faclor * o(l(T 9 ) to o(l O ' 1 )• It is 

to be expected (as somewhat analogous to human intelligence tests) that the intelligence and elegance metrics will 
occasionally need to be re-normed with improved mission simulation and autonomous system development results. 
Correspondingly, the degrees of inaccessibility, D I? degrees of resistance, D R . and degrees of interaction, D IN , can 
also be “normalized” to range between the values of 0 to 10 and engineering judgment used in place of detailed 
analysis and mission simulation to define these parameters. 
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In the most global sense, mission success, risk, and costs are functions of not only the vehicle LOA, but also the 
vehicle’s intelligence , i , and elegance , s, at a given LOA. Mission success, as noted earlier, can only be assessed in 
terms of flight test and simulation. On the other hand, first-order parametric expressions can be posited for the 
functionality of risk and cost with respect to a vehicle’s intelligence and the elegance of its autonomous system 
implementation. Specifically, for the purposes of this exercise, it is conjectured that the functional forms for the risk 
and cost metrics are, respectively, as follows: 


Where 


And 


Risk « x(T)f(i *,£* ) 
a«^y(r) g (/,/)(i+K) 


(5a) 


(5b) 


N A*T i * V ’ 

f(h £ ) — ) 


* + («T 


g(i,£)=c l (i , y (s)' 


(5c) 


(5d) 


<T) = 


\_MTBF Vehlclt , + MTBF Payload + MTBF Prmiding 

Service/Data ^ 


(5e) 


y(T)=(C v + C m 



MTBFrr 


(5f) 


Note that T is the mission duration. The “non-autonomy-related vehicle, payload, or service interruption risks are 
encapsulated in the functions x(T) and y(T), which are directly proportional to mission duration, T. In effect, Eq. 
5a-f can be considered a generalization of early work for aerial explorers and planetary robotic systems 1,18 . In 
particular, note the parallel between Eq. 5e, intended for terrestrial HALE-type UAVs, and Eq.4c for planetary aerial 
explorers. In other words, the probability of sensor failure is analogous to the mean time between failure (MTBF) 
for the payload system(s). The mean probability of unsuccessful sensor dispersal is analogous to the MTBF of the 
vehicle (and/or mid-air-deployment of surface interactive payloads). And, finally, the mean probability of 
unsuccessful data relay to Earth is analogous to MTBF related to service interruption for a terrestrial UAV. The 
coefficients ai, a 2 , a 4 , n! to n 3 , Ci, mo and m 2 in Eq. 5a-f are initially assumed to be equal to unity (i.e. have a value of 
one). Note though that a 3 «a 2 , in this case then a value was assigned where a 3 =0.005; this constant in the proposed 
risk function, Eq. 5c, accounts for the possibility that the vehicle might unnecessarily be too “intelligent” for its 
intended LOA and mission application. These constants will need to be validated in the future with acquisition of 
vehicle/mission data from actual aircraft operational experience or mission simulations. Such a formulation suggests 
that there is an optimal combination of aerial vehicle intelligence and elegance for a given LOA and mission 
application to be sought by means of system analysis accounting for vehicle autonomy. 
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Figures 5 and 6 illustrate the proposed trend of the risk and cost metrics, as a function respectively of UAV 
normalized intelligence and autonomous system implementation elegance , at a given LOA. As can be seen in both 
Figs. 5 and 6, increasing elegance, in general, reduces risk and cost. Increasing vehicle intelligence reduces risk 
(except for the slight increase at the high values to reflect potential unnecessary levels of intelligence) but at a price, 
increased vehicle cost. 



Figure 5. Trend behavior of proposed risk function, f(i,c) 



Figure 6. Trend behavior of proposed cost function, g(i,£) 
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C. Autonomy, Operational Staffing & Ground Control Costs 


Referring back to Table 1, the trade-off between vehicle autonomy and the amount of human operator “hands- 
on-time,” H 0 , and the number of operators per aircraft, N 0 , can (using the VSP HALE Sector autonomous operations 
goals and LOA definitions) be expressed in terms of the vehicle level of autonomy, K, as 


Hq = = CLq + CL ^ + CL^*$ + CL 

N 0 =b 0 - [V(K - 2) + b 2 u{tt - 5)] 

(6a-b) 


Where the coefficients are given by a 0 =1.0, a^-0.119, a 2 =-0.102, a 3 =0.018, b 0 =2, bi=l, and b 2 =0.5; the Eq. 6a 
polynomial curve-fit coefficients are derived from least-squares regression of the Table 1 LOA operator “hands on 
time” limits. The function u(x) is the unit step function, where if x is less than zero then u(x)=0, else u(x)=l. 
Figure 7 illustrates the operator hands-on-time trend line. 



In order to estimate the consequences of UAV autonomy, intelligence, and elegance on overall personnel 
requirements, and not just vehicle operators/pilots, it is necessary to define a baseline operational “staffing” matrix, 
M, such that 


#of Vehicle Operators 
#of Flight Engineers & Monitors 
# of Payload Specialists 


# of Mission Planners/Flight Directors 


(7a) 


As a baseline, an operational “staffing” matrix makes a certain assumption as to level of autonomy, i.e. = 2 . 
This baseline operational staffing vector is populated by constants that are based on generic and/or approximate 
characteristics of UAVs currently flying. For the purposes of this analysis, the M vector is populated as follows 
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K 



\fi . J 


(7b) 


Where the constants in the vector M are pi=l, p 2 =l, and p 3 =l. Note that one might assume that the back-up 
operator could as act as back-up flight engineer/monitor, however this individual is book-kept only under the 
operator staffing. Alternate crew staffing can be implemented by changing the values of M. 

Now for a given LOA, intelligence, and elegance combination, the following staffing level can be estimated 
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( 8 ) 


The above relationship holds true for 0<i<4; when i=0 then Eq. 8 devolves to Eq. 6b. The newly introduced 
constants are arbitrarily assigned to be b 3 =l and b 4 =l. Note that N and M are in terms of number of operational 
personnel per aircraft per operational shift. Equation 8 presupposes a priority phase-in of autonomous system 
technologies into other operational roles beyond that of operator/pilots. 

The HALE UAV maintenance manpower required is handled in a separate manner from that of the operational 
staffing estimation, but the maintenance support estimates are also affected by autonomy considerations. In this 
regard, the conventional approach (using a nominal maintenance man hour per flight hour, MMH/FH, multiplier) to 
estimate maintenance labor costs 8 is modified as follows 


MMH Kie+b s 
FH ~fc 6 X/V + l 

( 9 ) 

Where the constants are b 5 =18 and b 6 =l/9 (assuming a 50% reduction in maintenance labor costs as a consequence 
of implementing high levels of autonomous system technology). 

Finally it should be noted that the total number of operations staff per aircraft per operational shift is given by 
the simple identity (where N is given by Eq. 8) 


n - IX 

( 10 ) 


Figure 8 illustrates the operational staffing trend as a function of vehicle level of autonomy. 
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Figure 8. Operational staffing 


One UAV-unique technology that sees cost growth with increasing implementation of autonomous system 
technology is the ground control station, at least at the low- to mid-ranges of such technology. At the highest level 
of technology, where the vehicle basically runs itself, the need for ground-control sophistication decreases and costs 
will correspondingly drop as well. 


^gcs ~ “oGW 


ttl 8 


h(K-4)(K/*) + a; 


( 11 ) 


The basic functionality of the ground control station cost is to assume a conventional cost estimating relationship 
(CER) approach of dollars per pound of vehicle gross weight, GW. As a baseline, the proportionality constant is 
assumed to be ao=$1000/lbf (approximately half that of the vehicle avionics cost per pound value 8 ). However, this 
simple relationship is also modified by a term accounting for some dependence on level of autonomy. Initially, it is 
presupposed that including autonomy and overall intelligence into the HALE UAV increases the cost of the control 
station (employing the logic that a complex system needs another complex system to monitor it, if nothing else); 
however, at some breakpoint, the autonomy and intelligence of the vehicle becomes so great that the need/ability to 
monitor/control the vehicle becomes less demanding and the control station costs begin to decrease. (The most 
extreme example would be a PFV in which no real-time control is exerted over the vehicle and monitoring consists 
primarily of one way data capture from the vehicle.) Note however that one might consider that increasing the 
elegance of the autonomous system on the aerial vehicle would reduce the cost of the control station, as a simpler 
but more robust system needs a less complex system to monitor it. Note that a constant value of ai=25 is suggested. 


D. Relationship between Mission Requirements and Autonomy Requirements 

Drawing upon the Table 2 mission descriptions and the Table 3 mission success metrics, Table 4 summarizes an 
assessment, using engineering judgment at this point of the analysis, as to key operational/environmental parameters 
that will be used shortly to postulate functional relationships between these parameters and metrics for HALE 
platform intelligence and elegance. 
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Table 4. Functional Relationship Assessment 


Notional HALE Mission 

Planetary 
(F=l) or 
Terrestrial 
(F=0) 

Duration, 

T (Days) 

Alt. 

(kft) 

A 

(1-10) 

A 

(1-10) 

Av 

(1-10) 

d-5) 

(1-10) 

Telecom Relay Platform 

0 

30 

60 

1 

2 

2 

4 

2 

Storm Tracker 

0 

4 

40 

3 

7 

3 

4 

4 

Border and Littoral Water 
Patrol 22 

0 

4 

40 

4 

2 

3 

4 

3 

Station-Keeping Science 
Platform 

0 

30 

40 

3 

1 

2 

4 

3 

Long-Range Science Platform 

0 

4 

40 

3 

5 

4 

5 

4 

Sensor-Network Aerial 
Constellation 

0 

4 

25 

4 

3 

7 

5 

4 

Wildfire Surveyor 

0 

1 

25 

3 

3 

2 

3 

3 

Aerial Explorers 









• ARES -type prescribed flight 

paths 15,19 

1 

0.04 

3.28 

2 

2 

1 

3 

3 

• Search & Find 
Behaviors 11,13,14 

1 

0.33 

1.64 

4 

4 

2 

4 

5 

• Surface Interactive - robotic 
symbiotes 12 

1 

30 

0.66 

6 

6 

7 

5 

7 

• Surface Interactive - 
VTOL 18 

1 

90 

0.033 

7 

8 

8 

5 

8 


As can be expected, the Table 4 values are only approximate descriptions of a broad range of possible mission 
types. Resulting functional relationships have been derived on the basis of functional analysis using the Table 4 
“data” - which in turn was derived using collective engineering judgment. Refer to the following: 


i* =f(F,T,Alt,D„D R ,D IN ,K) 


Or, more specifically 


i ~ a 0 u(F — l) + (l + — 2) + w(b> — 5)) + a 2 Dj + a 2 D R 

■ m 2 ! Alt 


+ Cl A 


( A/v ^ 


l-e 


+ a 5 l-e 


+ cl. 




T + 1 


(12a) 


(12b) 


Where the following constant values are suggested (so at to arrive at agreement with the Table 4 results): ao = 1, ai = 
0.5, a 2 = 0.25, a 3 = 0.25, a 4 = 0.5, a 5 = 1, and a 6 = 0.5; mi = 0.5, m 2 = 1000, and m 3 = 10. Note that constants m 2 and 
m 3 are relatively large and are used to scale T and Alt. Figure 9a-f illustrates the incremental normalized 
intelligence metric trends with the key independent parameters. 
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Figure 9a-f. Incremental Vehicle Normalized Intelligence Contribution from Key Mission Parameters: (a) 
LOA, (b) D I? (c) D r , (d) D IN , (e) cruise altitude, & (f) mission duration 


E. Individual Autonomous System Technologies & Achieving Autonomy Metrics & Goals 

Relating the development progress and contribution of individual technologies, represented by the j th array 
element B* , to the overall intelligence metric for autonomous aerial vehicles is performed as follows 


1 w* + w: 


for j>l 


(13) 


Where S is a normalized (S<1) “mission success” metric derived from mission simulations incorporating the 
individual autonomous system technologies in specified set of vehicle/mission scenarios. Note that W* and W* 
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comprise a set of weighting factor metric for relative weighting given to the two types of technology assessments 
embodied in Eq. 13. The weight Wj is given to the “objective” simulation-derived mission success technology 
assessment. The weight W* is given to the technologist’s “self-assessment” of the normalized technology 
readiness level (TRL) of the jth autonomous system technology implemented in the mission simulation, , or as 
otherwise denoted by 


TRL of the 1st Technology divided by "9" 


B = 


TRL of the jth Technology divided by "9" 


(14) 


Note by definition that Wq + W* = 2 must hold true for all technologies, i.e. all values of j. The weights Wy are 

specified as follows, relying on the matrix Q, which in turn is derived from the QLD-inspired tabular matrix shown 
in Lig. 10. 


W j-dlj 


and 


I , = u 


Vt; * A 

10 —^ -+i -9 


(15a-b) 


Where 


W F = max(W ) 


and 




(15c-d) 


The constant a, Eq. 15a, is arbitrarily assigned to reflect the relative weight of W* with respect to W 0 *; a=1.0 is 

suggested. The array W can be thought of as denoting the relative importance of each individual technology, based 
upon the Q matrix input, as to contributing the overall goals of the project. This will be discussed further in the 
analysis and results section. Note that during the course of flight tests and/or mission simulations (embodying the 
autonomous system implementation), if an individual technology is not implemented then the “TRL” value of that 
technology is set to zero (irrespective of its previously demonstrated, but not implemented in the current simulation 
or flight test). 




Autonomous System Technologies 
1 0 

Goals: 

Objectives: 


1 

1 




Q Matrix of Weights 

M 

N 



Figure 10a. General format of QFD-Inspired Tabular Matrix 
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Finally, the progress towards developing individual autonomous system technologies needs to be tracked 
against progress towards overall programmatic goals/objectives (in so far as affected by those same autonomous 
system technologies, and not other “extraneous” vehicle technologies). This is accomplished in the proposed 

analysis by the following relationship for “fractional anticipated contribution to goal” array, C,- . 


Where 


c:-<yzQ,„. 

C = QB* 


(16) 


(17) 


Note that by definition, for a given i'th row of the matrix Q, the following holds true 


ZQ.j - 1 


( 18 ) 


It is appropriate to examine these notional vehicle/mission autonomous system capabilities in the context of a 
matrix inspired by Quality Function Deployment (QFD) “house of quality” matrix representation 3 " 6 - refer to Fig. 
10. The contents of this matrix naturally flow from the goals, objectives, technical challenges, and approaches (as 
derived by the well-known “GOTChA” process) developed and vetted by the VSP HALE Sector with appropriate 
review from NASA independent reviewers, industry and academia. 7 Note that the matrix columns represent the 
definition of a detailed technology capability set — as affected (and only affected) by autonomous system technology 
— consistent and supportive with respect to the HALE-sector-team-identified technical approaches. 

Note that the initial weighting factors, Qy, used in the Fig. 10 QFD-inspired matrix can be determined as per Eq. 
19. This initial weighting factor scheme assumes that contributing autonomous system technologies all 
uniformly/equally contribute to the i'th Goal with the nominal partitioning between enabling (A=l) and contributing 
technologies (A=0.25). 


Q, 


A 

Number of Contributing Autonomous System Technologies, including j' th technology, to ith Goal 


(19) 

Subsequent iterations on the Fig. 10 weighting factors can be adjusted to reflect sensitivity analysis results and/or 
mission simulations that show that autonomous system technologies do not uniformly contribute to the goals. 
Additionally, the weighting factors can also reflect resource/funding issues that may not fully stem from engineering 
considerations alone (i.e. not all promising technologies may be funded at the required levels, or funded at all, to 
achieve the anticipated contributions to the technology goals). 


F. Other Considerations in UAV Autonomy 

Finally what is also required to help evaluate UAV autonomous system capabilities is essentially a “Turing 
Test” 9 for autonomous aerial vehicles. Such testing would have to be conducted by means of extensive mission 
simulations of the vehicle under the guidance of its autonomous system software. Such autonomous vehicle mission 
simulations would also have to be benchmarked against missions “flown” with pilots/human-operators in the loop. 
In turn, scoring criteria for such testing could be based upon 1. overall mission success metrics (defined as in Table 
3) and 2. by “handling quality” metrics similar to the well-known Cooper-Harper 10 pilot ratings used for manned 
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aircraft. Autonomous aerial vehicles would be considered to have successfully passed this “UAV Turing Test” if 
the aggregate mission success and handling qualities for the autonomous aerial vehicle matched or exceeded the 
equivalent metrics for missions conducted with pilots/human-operators in the loop. 


Table 5. Notional “UAV Turing Test” Comparison 


Flight Phases 

Mission ”X” -Autonomous 

Mission “X” - “Piloted” 




Take-off (with x% probability - Gaussian 
distribution - of runway abort) 






Navigating and flying waypoint-to- waypoint 
trajectories within prescribed precision 






Landing (with z% probability of final 
approach abort) 




Alternatively, a third-party knowledgeable observer could provide the “UAV Turing Test” ratings of whether a 
vehicle is autonomous or “piloted” (i.e. “check” the boxes in Table 5). This observer/rater would also have the 
additional role of being able to override the scripted mission scenario and instigate failure modes and change of 
flight profile/plans. If the majority of tasks are rated as “piloted” by the observer, when in reality the 
vehicle/simulation is fully- or semi- autonomously controlled, then the vehicle/simulation “passes” the “UAV 
Turing Test.” In this regards, this “UAV Turing Test” approach is more consistent with Turing’s original 
“imitation game” proposal. 9 

Finally, as was noted earlier, system analysis of autonomous systems should consider and address first-order 
concerns regarding unique UAV operational hazards in order to examine reasonable trades between the vehicle 
LOA, intelligence, and elegance. Therefore, the appropriate questions are as follows. What then are the unique 
consequences of the autonomous system failure, versus human operator error, for the UAV, the mission, and 
collateral human/property damage? A corollary to this question is, are there unique hazards for autonomous HALE 
UAVs versus (remote -piloted) human operation, or is it merely a question of relative respective probabilities of 
occurrence for a commons set of hazards? The short answer is that both occur - there are some unique hazards as 
well as many other hazards where it is only a matter of relative probability between a hazard seen by a piloted 
(human-occupied) versus autonomous aerial vehicle. This issue is further complicated by the fact that many manned 
aircraft have, or will have, highly automated sub-systems onboard them, resulting in a grey area as to what is a 
“unique UAV hazard,” at least as far as autonomous systems technologies are concerned. Table 6 is a representative 
subset of the operational hazards associated with manned aircraft versus uninhabited aerial vehicles, as influenced 
by autonomous system technologies. 
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Table 6. Uniquely Human versus Autonomous System Operational Hazards 


Category 

Probability 

Hazard 

Hazard Control(s) 

Uniquely Human Hazards — 

I 

See note 
below 

Errors in judgment due to human 
(pilot, engineer/tech support 
staff, payload specialists, etc.) 
fatigue or work overload. 

1 . Implement ergonomic human-factor friendly 
interfaces and control and decision-making tools 

2. Implement balanced portfolio of autonomous system 
technologies on the vehicle and in ground-control 

II 

* 

Mission failure, or reduced 
operational value (e.g. missed 
opportunities), as a consequence 
of inadequate response time for 
human mission planners to 
discus s/achieve consensus. 

1. Enhance/augment human mission planners with 
intelligent mission management tools 

2. Implement intemet-like data/information networking 
to maximize efficiency of the mission planner decision- 
making process (making sure all vested interested are 
represented in real-time despite geographic distribution) 

I 

* 

Mission/vehicle loss, or reduced 
operational value, due to 
inadequate reaction time for the 
pilot to interpret advanced (non- 
visual) sensor suite input to 
maintain adequate vehicle 
control 

1. A key justification for the use of autonomous UAVs 
for high-risk military or public missions 

2. “Pilot-in-the-loop” simulations examining sensor 
fusion and display(s) during relevant mission scenarios 
including adverse weather conditions (e.g. low- visibility) 
and/or extreme operating conditions (e.g. high-speed, 
low-altitude flight). 

II 

See note 
below 1 ^ 

Mission abort/failure under 
special circumstances where 
vehicle (& occupants) can not be 
treated (as needed) as 
expendable 

1 . Another key justification for use of UAVs in high-risk 
military or public good missions 

2. Costly reliance on other assets (e.g. ground troops, 
artillery, and air- suppression for military missions) to 
reduce aircraft and aircrew risk down to acceptable levels 





Unique Autonomous System Hazards — 

I 

* 

Vehicle, or mission loss, due to 
improper/inadequate software 
programming & testing 

1 . Adoption of formal software validation and 
verification processes 

2. Rigorous “hardware-in-the-loop” ground-testing 

3. Extensive mission simulations (especially when 
mission includes flight in unknown/uncertain 
environments and/or non-deterministic autonomous 
decision-making processes are employed) 

II 

t 

Improper execution of flight 
behaviors/tasks as a consequence 
of using nondeterministic 
(stochastic or heuristic) vehicle 
decision-making processes 

1. Use non-deterministic processes (hardware/software) 
when only absolutely essential to assure mission success 

2. Tightly/determini Stic ally constrain the limits of action 
(task execution) that can be effected by non-deterministic 
processes (i.e. “idiot-proof’ the actions/controls that are 
commanded/tasked by the nondeterministic processor) 

II 

t 

Improper implementation of 
adaptive flight control 

techniques (with either micro- 
flow control actuators or 
conventional control surfaces), 
especially under highly degraded 
vehicle operating conditions 

1. Extensive ground-test and wind-tunnel testing; 
coupled with flight test envelope expansion that carefully 
examines adaptive flight control techniques/actuators - 
including simulated vehicle/actuator failures and 
performance degradation 

2. Incorporation of high (& costly) risk margins for 
aircraft hardware/systems as a contingency to restrained 
use of adaptive flight control techniques 

I 

* 

Flight load limit excursions, or 
unanticipated system failures, 
because of an improperly 
functioning intelligent vehicle 
system management system. 

1 . Adoption of formal software validation and 
verification processes 

2. Rigorous “hardware-in-the-loop” ground testing of 
intelligent vehicle system management systems (e.g. load 
limiting avionics, health monitoring and prognostics). 
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Note, the following are definitions of the above severity categories and hazard probabilities: 
Category I. “catastrophic,” II. “critical,” III. “marginal,” and IV. “safe.” (or, rather, let C=(5-l)=4, 
(5-2)=3, (5-3)=2, and (5-4)=l); Probability levels of “probable,” “remote,” “improbable,” and 
“highly improbable” (or a value of P is assigned such that it falls within the discrete ranges of 
0.1<P<1.0, 0.01<P<0.1, 0.001<P<0.01, and 0<P<0.001 - which is more consistent with an 
experimental flight test program and not production aircraft). Further, note that: approximate 
probabilities can be assigned on the basis of existing (though perhaps with limited operational 
experience) manned and UAV aircraft, and ^hese probabilities can only be assessed as 
developmental experience with individual autonomous system technologies (for a given vehicle- 
type and mission) are gained. 


The Table 6 hazards are at a fairly high level. Ideally, each of the individual technologies identified in the Fig. 
10 tabular matrix should have its own one or more specific hazards associated with it. The results of such an 
expanded detailed technology/hazard analysis can be recast to the following matrix form for subsequent analysis. 


1st Hazard ... 

jth Hazard ... 

1st Technology C P 


ith Technology 

CP 




( 20 ) 


Given the above, the individual technology hazard rating, accounting for technology maturation/readiness, is given 
by 


» -I» « 

( 21 ) 

Where the individual developmental progress made towards an (i'th) autonomous system technology, B. , is given, 

as earlier, by Eq. 13. The smaller the value of H. , then the smaller the overall hazard for a given autonomous 

system technology, at its current demonstrable (at least in terms of simulation, that is) technology 
readiness/maturation. 

Finally, to complete this discussion of vehicle/mission hazards, it is important to note that the cost/risk trade 
between using manned aircraft versus UAVs for identic al/similar missions is an important area of investigation but 
outside the scope of this paper. Such cost/risk trades inevitably have to address cost metrics related to the potential 
loss of manned aircraft (with attendant loss of crew and passengers) which, in turn, would need to address both 
“actual” versus “perceived” costs and benefits. These types of trades studies are really only applicable for military 
missions or high-risk, public-good, life-saving disaster-relief or other emergency missions where risk to the vehicle 
(and crew if applicable) must be traded against the “higher good” of rescuing fallen comrades or saving lives. And 
yet, UAVs may find their greatest utility under just these circumstances. As such, future system analysis work 
should be devoted to examining these types of issues. 
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II. Analysis and Preliminary Results 


The above outlined methodology for incorporating autonomous system technology considerations into aerial 
vehicle system analyses is very conjectural at this current stage of development. Validation of key assumptions and 
functional trends clearly needs to be performed as a next step. 

The HALE technologies being developed by the NASA VSP are focused on enabling new mission capabilities. 
A mission level perspective is necessary to truly capture the importance and impact of these technologies. Short of 
conducting actual mission demonstrations, the best way to analyze the role of these new autonomous systems 
technologies in contributing to mission success is through mission simulations. Mission simulation capability is 
especially important in the case of autonomous systems technologies where it is not the traditional aircraft 
performance metrics (speed, range, endurance, etc.) that are impacted but mission level performance. Simulation 
modeling needs to be developed to assess the performance of HALE and PFV vehicles and technologies in "real 
world" application so as to both demonstrate the overall mission capabilities and track the benefits of technologies 
under development. 


Some initial insights into the optimum level of vehicle autonomy and intelligence required for a particular HALE 
UAV mission/application can now be made, given the work presented so far. As noted earlier, whatever attempt to 
optimize vehicle performance in the context of vehicle autonomy and intelligence has to also allow for acceptable 
levels of success and risk while at the same time keeping development and implementation costs to a minimum. 
Figures 11 and 12 illustrate the general return on investment trends for autonomous aerial vehicles performing 
commercial mission applications. The MROI trends are shown as a function of mission duration and whether or not 
mission success is fixed-rate revenue (as a function of time) or fixed price per mission. Commercial HALE UAV 
“mission return on investment” will always be related ultimately to revenue and dollars. MROI, in this case, is 
relatively easy to interpret. The MROI concept is somewhat harder to interpret when the return is not in terms of 
revenue but scientific information/opportunities. 



Normalized Intelligence, i* 

Figure 11. Return on Investment Trend as a function of Mission Duration (without cost penalty for increased 
launch/recovery frequency for shorter duration missions & LOA = 4) 
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One of the key observations to note in Fig. 1 1 is that there is an optimum peak in MROI with respect to the vehicle 
normalized intelligence metric; this is driven by the associated increase in risk and cost in having too much 
intelligence unnecessarily built into the vehicle to accomplish its given mission. The MROI trend results 
(accounting for risk as well as cost) for Fig. 1 1 do not incorporate a cost penalty for increased launch/recovery 
frequency for shorter duration missions. As such the Fig. 1 1 results show the shorter-duration aircraft (with more 
sorties) having higher MROI than the longer-duration aerial vehicle; this is direct consequence of the anticipated 
higher development costs and risks (associated with decreased reliability over longer time) for longer-duration 
vehicles. The MROI trend for the shorter-duration aircraft should shift downward somewhat when the cost penalty 
for launch/recovery frequency is accounted for. 



Normalized Intelligence, i* 


Figure 12. Mission Return on Investment Trends: Mission Success is directly proportional to (a) Mission 
Duration and (b) Fixed Mission Revenue (mission duration = 10 Days and LOA = 4) 


Figure 12 illustrates the influence of mission success (and its overall functionality) on MROI — in this case in the 
context of fixed-rate revenue which is dependent on mission duration and a fixed-price per mission revenue. In this 
particular case, Fig. 12, the fixed-price per mission charged is too low and is not competitive with respect to fixed- 
rate per day pricing. The decision to pursue fixed-price versus fixed-rate pricing for commercial HALE UAV 
missions will be strongly influenced by the risk assumption for successfully sustaining over the complete planned 
duration of the mission. 

Ultimately the cost associated with “autonomous system technology” development and implementation is only 
cost-effective in the context that reductions in operational staffing costs and maintenance manpower costs are 
sufficient to offset the increased vehicle development costs attributable to vehicle autonomy. Figure 13 and 14 
present cost estimate trends for HALE UAV missions. 
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Figure 13. Cost Trend Estimates (LOA=4, Elegance=5, Aircraft Projected Life = 10 years, and Aircraft 
Fractional Usage = 0.66) 

Figure 13 presents the incremental costs associated for aircraft and mission, the ground control station (GCS), 
the operational staffing costs (in lieu of manned aircraft crew costs), and aircraft maintenance manpower. Next to 
the vehicle costs, maintenance manpower costs dominate - followed by operational staffing and GCS costs. Longer 
duration aircraft will create significant technical challenges to achieve the necessary reliability levels; achieving 
such reliability will dictate new autonomous system technologies with nontrivial development costs. 



Figure 14. Operational Staff and Maintenance Manpower Cost Trends as a function of Aircraft Projected 
Life (baseline trends: i*=e =1; minimum trends: i =c =10) 
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Figure 14 presents the cumulative operational staffing and maintenance manpower cost trends as a function of 
aircraft life. The difference between the “baseline” and “minimum” cost trends represents the potential cost savings 
for implementing pertinent autonomous system technologies. 

What are the specific attributes of an autonomous system implementation essential for a given 
mission/application and aerial vehicle in order to maximize mission success? The results in Figs. 15-18 are drawn 
from, and are therefore consistent with, the numeric data contained within the QFD-inspired tabular matrix, Q, 
shown in Fig. 10, given Eq. 19. The resulting distribution of Fig. 15 is a consequence of both the usage of uniform 
weighting of technologies contributing to an individual goal, as well as relative scaling of “enabling” versus 
“contributing” technologies within Q. 
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Figure 15. Prescribed relative importance of individual autonomous technologies 


Given the uniform weighting used in the Fig. 10 tabular matrix, the most important (relatively speaking) are the 
technologies associated with the development and control of optimized multifunctional structures, and optimized 
and/or active-control strategies for vehicle aero-performance. Given the Q matrix derived from Fig. 10, the (number 
of funded) technology portfolio - as a function of the vehicle normalized intelligence metric - is shown in Fig. 16. 
A more monotonic trend is desirable from a technology portfolio perspective - versus the near-exponential trend 
seen in Fig. 16. This will, in turn, dictate a future revisiting of the imposed uniform weighting in the Q matrix. 
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Figure 16. Portfolio assessment of technologies as a function of normalized intelligence metric target 


Given an autonomous system technology portfolio assessment such as summarized in Figs. 15-16, running 
assessments can be made (based on concurrent mission simulation results) as to individual technology’s anticipated 
fractional contribution to the goals (or more correctly the objectives intrinsic to the goals). Figures 17 and 18 show 
notional results for such a fractional contribution to goals assessment for two levels of self- and objective- estimates 
of individual technology progress. Figure 17 shows the progress made on an initial portfolio of high-priority 
technologies towards the vehicle/mission goals given a relatively low TRL and mission success metrics from early 
technology development efforts and mission simulation results. In Fig. 17 there are clear examples of where goals 
are not being met at the current normalized intelligence metric design target and the current self- and objective 
assessments of technology readiness/progress. 
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Figure 17. Progress towards objectives and goals (uniform technology self-assessment of TRL=4, mission 
simulation measured normalized mission success of S=0.25, and normalized intelligence metric target of i =7) 
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Figure 18 illustrates a fractional contribution to goal assessment based on a notionally somewhat larger (in 
terms of a higher-priority portfolio) and more mature set of autonomous system technologies (from the Fig. 10 Q 
matrix). In this example, goals are being more uniformly and consistently met. There are still, though, clear 
examples of where improved technological progress could be achieved relevant to the goals. 


E3 


S Sh 

S eg 


s H 

I s 

O 

a a> 
© > 

3 o 

© O 


‘-3 © 

II 

13 E 

a © 
o o 

u 



Individual Technology Objectives (Mapping to Goals) 


Figure 18. Progress towards objectives and goals (uniform technology self-assessment of TRL=4, mission 
simulation measured normalized mission success of S=0.5, and current normalized intelligence metric target 
of i*=9) 


This concludes a very brief summary of the type of system analysis that can be performed in support of 
autonomous system trade studies for HALE UAV platforms. Considerably more work remains to be performed - 
particularly in terms of validation and detailed mission assessments. This work will hopefully be recognized as an 
important first step in a new sub-domain of system analysis. 


III. Concluding Remarks 

The first step in solving a problem is to pose the right questions to ask and to investigate. A set of such 
questions has been posed in this paper for preliminary investigations into the impact of autonomous system 
technology on uninhabited aerial vehicles and HALE missions in particular. Additionally, a preliminary 
methodology for system analysis incorporating/considering autonomous system technology issues into the study of 
overall aerial vehicle capabilities and mission requirements has been outlined. In order to perform this system 
analysis it was critical to define autonomy metrics that were both quantifiable and practical. Additionally, first order 
functional relationships were proposed that related these autonomy metrics to vehicle characteristics and mission 
requirements. Finally, the contribution of individual autonomous system technologies (of which a large set were 
identified) were related in both a conceptual as well a quantitative sense to high-level programmatic goals, mission 
cost, risk, and return on investment criteria. All of this work is still fairly conjectural at this stage, though. 
Considerable work remains in order to arrive at a validated set of system analysis tools for UAVs and other 
automated and intelligent systems. 
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